Tools for a sources mirror #9341
Open
+320
−5
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Ian Leonard <antonlacon@gmail.com>
Signed-off-by: Ian Leonard <antonlacon@gmail.com>
Signed-off-by: Ian Leonard <antonlacon@gmail.com>
Signed-off-by: Ian Leonard <antonlacon@gmail.com>
|
6270 works if you're only interested in downloading sources for one device. If this merges, I may extend build-mirror to use parallel for multiple downloads and then close it. I'm not expecting 6270 to merge. 8279 is complimentary. It shares some logic in figuring out what files are needed, but it's meant for pruning a builder's sources directory, instead of populating a mirror. I don't remember why it's marked draft, but I'll revisit it soon. Long term it might make sense to merge it and genmirrorlist.py together, but I'm not in a rush and finding reviewers for python is already tricky. |
Signed-off-by: Ian Leonard <antonlacon@gmail.com>
Signed-off-by: Ian Leonard <antonlacon@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds tools for building a sources mirror. Its broken into three parts:
genbuildplan.py: modified to add command line arguments for use by other scripts
genmirrorlist.py: new script to output a list of $PKG_NAME and $PKG_URL, one per line, of the requested images to be built. May be invoked by setting PROJECT/DEVICE/ARCH, invoking
--all, or invoking--allwith--builddirs.--allgenerates a list for all images (takes 20-30 minutes to put together the list - if the uboot based images don't vary in their package needs, this can be reduced significantly).--builddirsreduces--allto only those images with a build directory present. PROJECT/DEVICE/ARCH is for that specific image. Script outputs to stdout or to a file with the--exportoption.build-mirror: standalone shell script that will download the output of genmirrorlist.py into the specified output directory with $PKG_NAME subdirectories. This is separated from genmirrorlist.py because genmirrorlist.py needs to run in the git source tree, and this doesn't. Add
-vto show download progress.Examples:
PROJECT=RPi DEVICE=RPi5 ARCH=aarch64 tools/genmirrorlist.py | tools/build-mirror -o mirror/tools/genmirrorlist.py --all --export package-list-20241001.txt && tools/build-mirror -i package-list-20241001.txt -o mirror/